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System And Method for Displaying Seamless Immersive Video 

Paul A. Youngblood 
Vlad Margulis 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to immersive video 
systems, and specifically to a system and method for displaying 
immersive videos. 

Discussion of the Related Art 

[0002] Immersive videos are moving pictures that in some sense 
surround a user and allows the user to "look" around at the 
content of the picture. Ideally, a user of the immersive video 
system can view the environment at any angle or elevation. A 
display system shows part of the environment map as defined by 
the user or relative to azimuth and elevation of the view 
selected by the user. Immersive videos can be created using 
environment mapping, which involves capturing the surroundings or 
environment of a theoretical viewer and rendering those 
surroundings into an environment map. 

[0003] Current implementations of immersive video involve 
proprietary display systems running on specialized machines. 
These proprietary display systems inhibit compatibility between 
different immersive video formats. Furthermore, the use of 
specialized machines inhibits portability of different immersive 
video formats. Types of specialized machines include video game 
systems with advanced display systems and high end computers 
having large amounts of random access memory (RAM) and fast 
processors . 

[0004] Figure 1A is a representation of a 360° immersive 
picture P_l, i.e. an environment map. The entire field of view 
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in immersive picture P_l shows a tree TREEl , a house portion 
H0USE1„A, a house portion H0USE1_B, and a full house H0USE2 . 
Because memory is arranged in a two-dimensional array, immersive 
picture P_l is stored as a two-dimensional array in memory. 
Thus, the data along edge El is not directly correlated to the 
data from edge E2 . As a result, house portions H0USE1_A and 
H0USE1_B, which in the environment of a centrally located 
theoretical viewer (not shown) are joined into a full house 
H0USE1, are instead separated when immersive picture P_l is 
stored in memory. Immersive pictures, such as 3 60° immersive 
picture P_l, should represent a three-dimensional (e.g. 
cylindrical) space. As a result, in displaying immersive picture 
P_l, the two-dimensional representation of Figure 1A must be 
converted to a three-dimensional representation. 

[0005] Figure IB is a cylindrical representation of immersive 
picture P_l of Figure 1A. Seam S_l is formed from joining edges 
El and E2 together to form this cylindrical representation from 
the two-dimensional representation of immersive picture P_l shown 
in Figure 1A. When edges El and E2 are joined as shown, house 
portions H0USE1_A and H0USE1„B are joined into full house H0USE1 . 
Thus, seam S_l runs through full house H0USE1 and is the dividing 
line between the house portion H0USE1_A and the house portion 
HOUSE 1_J3. Tree TREEl, located on the door side of house portion 
H0USE1_B, is also shown. 

[0006] Figure 1C is a conceptual cylindrical representation of 
the 3 60° immersive picture P_l of Figure 1A. The contents of 
immersive picture P_l are omitted for clarity. This conceptual 
cylindrical representation indicates the perception of a 
theoretical viewer looking at immersive picture P_l from the 
vantage point of a location VIEWPOINT, located within the 
cylinder formed by immersive picture P_l . Immersive picture P__l 
is a 360° immersive picture having a first edge El and a second 
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edge E2 . Similarly to Figure IB, seam S_l results from the 
joining of the two-dimensional representation (Figure 1A) edges 
El and E2 in the cylindrical representation. 

[0007] A view window 101 represents the portion of immersive 
picture P_l visible to the user at location VIEWPOINT. View 
window 101 is centered at the origin of a three dimensional space 
having x, y, and z coordinates, where z (not shown) is 
perpendicular to the plane of the page. Similarly, the 
environment surrounding the user located at the location 
VIEWPOINT is represented by the cylindrical representation of 
immersive picture P_l that is centered at the location VIEWPOINT. 
View window 101 is typically displayed on a display unit for the 
user of the immersive video system. Thus, only the portion of 
immersive picture „1 visible to the user, rather than the entire 
picture content, is displayed, for example, on a television 
screen . 

[0008] By moving view window 101 (e.g. left or right) relative 
to immersive picture P_l, the portion of immersive picture P„l 
visible to the user may be changed. This relative movement of 
view window 101 with respect to immersive picture P„l is called 
panning. By moving view window 101 clockwise 360°, the entire 
circumference of immersive picture P__l may be traversed. A 
cursor 102 within view window 101 is controlled by the user and 
indicates the desired direction of panning. Cursor 102 is 
located to the seam S_l side of view window 101 in Figure 1C . 
[0009] Figures ID and IE are a cylindrical representation of 
the 3 60° immersive picture P_l of Figure 1C rotated clockwise a 
first and second amount, respectively. Again, the contents of 
immersive picture P_l are omitted for clarity. Because cursor 
102 is located to the seam S_l side of view window 101, immersive 
picture P_l has panned clockwise with respect to view window 101 
from Figure 1C . 
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[0010] Figure IE shows seam S__l as visible within view window 
101. As described above, immersive picture P__l is stored two- 
dimensionally in memory, therefore, the data for edge El is not 
directly correlated to the data from edge E2 . As a result, when 
panning across seam S_l, the data from edges El and E2 must be 
joined before being shown to the user on a display as a whole 
picture. Because real-time picture display systems can't join 
images fast enough to display seams, it is preferable not to 
display seam S_l in view window 101. It would be desirable to 
have a method of panning across a picture having seams without 
real-time seam distortion visibly showing in the view window. 

[0011] Accordingly, there is a need to deliver an immersive 
video. experience across many different non-specialized platforms 
while minimizing distortion created by real-time joining of 
picture seams in the field of view. 



SUMMARY OF THE INVENTION 

[0012] In accordance with the present invention, an immersive 
video system is provided which enables a user to interact with an 
immersive video on a variety of platforms. To accommodate 
different types of components found on different platforms, the 
resolution of the immersive video may be changed to adapt to 
different amounts of random access memory (RAM) on a given 
platform. 

[0013] In one embodiment, a pair of cylindrically defined 360° 
immersive videos are simultaneously played in a standard display 
software program. These two immersive videos are created such 
that seams in one video are separated from seams in the second 
video by at least an amount equal to the length of the view 
window. The display software program can be chosen such that it 
is supported by a variety of platforms. For example, choosing 
Macromedia™ Flash as a display software program allows playback 
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on any platform supporting Flash. A view window associated with 
the standard display software program defines the portion of the 
immersive video shown to the viewer. A control mechanism 
adjusted by the user pans the view window around one of the pair 
of immersive videos. Panning is the act of moving a point of 
view in a particular direction (e.g. left of right). Because two 
immersive videos having different seams are simultaneously 
played, the view window may select to display a portion of the 
video without the seam. Thus, if the view window approaches a 
seam while displaying a portion of a first video, the view window 
is changed to display a similar portion of a second identical 
video that has no seam in that location. 

[0014] In another embodiment, a cylindrically defined 
immersive video representing an environment map larger than 360° 
(e.g. 420°) is played in a standard display software program. The 
overlapping portion of this immersive video (i.e. the portion 
greater than 360°) is used to avoid displaying picture seams (or 
picture edges) to the user. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] Figure 1A is a representation of a 360° immersive 
picture . 

[0016] Figures IB-IE are cylindrical representations of a 360° 
immersive picture. 

[0017] Figure 2A is a cylindrical representation of the 
coverage of two 36 0° immersive videos in accordance with one 
embodiment of the present invention. 

[0018] Figure 2B is a two-dimensional representation of the 
coverage of two 3 60° immersive videos in accordance with the 
embodiment of Figure 2A. 
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[0019] Figures 3A-3C are two-dimensional representations of 
the coverage of two 3 60° immersive pictures in accordance with the 
embodiment of Figure 2B. 

[0020] Figure 4A is a two-dimensional representation of an 
environment map larger than 360° in accordance with an embodiment 
of the present invention. 

[0021] Figure 4B is a cylindrical representation of an 
environment map larger than 3 60° in accordance with the embodiment 
of Figure 4A. 

[0022] Figures 4C-4E are cylindrical representations of an 
environment map larger than 360° in accordance with the embodiment 
of Figure 4B. 

[0023] Figures 4F and 4G are representations of two- 
dimensional time sequenced environment maps larger than 360° 
degrees in accordance with the embodiment of Figure 4A. 
[0024] Figure 4H is a representation of a two-dimensional time 
sequenced video environment map larger than 3 60° degrees in 
accordance with the embodiment of Figure 4A. 

[0025] Figures 41 and 4J are two-dimensional representations 
of two immersive pictures in the time sequence video environment 
map of Figure 4H. 

[0026] Figure 5 is a two-dimensional representation of a two 
time sequenced 3 60° immersive videos in accordance with the 
embodiment of Figure 2A. 

[0027] Figures 6A-6C are two-dimensional representations of a 
three pictures in two 3 60° immersive videos in accordance with the 
embodiment of Figure 2A. 

[0028] Figure 7 is a two-dimensional representation of two 
time sequenced immersive videos in accordance with an embodiment 
of the present invention. 
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[0029] Figure 8 is a block diagram of a system implementing an 
immersive video display system in accordance with an embodiment 
of the present invention. 

[0030] Similar elements in the above Figures are labeled 
similarly. 

DETAILED DESCRIPTION OF THE DRAWINGS 

[0031] In accordance with the present invention, a cross- 
platform immersive video system is described that allows panning 
during playback of an immersive video. The use of panning in 
conjunction with a moving picture allows a real-world, inclusive 
experience for the user. Multiple immersive videos (e.g. 2 
videos) are simultaneously displayed to compensate for distortion 
in the view window along video seams. Video seams are the point 
of combination of video filmed from two or more separate cameras. 
[0032] A standard display software program (e.g. Macromedia™ 
Flash) is chosen in conjunction with a specific platform (e.g. a 
standard PC) . The immersive video system is then adapted to 
requirements of that standard display software program. As a 
result, an immersive video system according to the present 
invention is made non-proprietary, thereby supporting the use of 
different platforms. This immersive video system is described in 
more detail below. 

[0033] Figure 2A is a cylindrical representation of two 360° 
immersive pictures in accordance with one embodiment of the 
present invention. Immersive picture P__2 is a 360° immersive 
picture having a first edge E3 and a second edge E4 . A seam S_2 
in immersive picture P„2 occurs where the edges E3 and E4 meet. 
Simultaneously played immersive picture P_3 is a 3 60° immersive 
picture having a first edge E5 and a second edge E6 . Similarly, 
immersive picture P_3 has a seam S_3 where edges E5 and E6 meet. 
Immersive pictures P„2 and P__3 are identical but for the location 
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of seams S_2 and S_3 with respect to the picture content. Seams 
S_2 and S_3 are separated by an overlap distance OVERLAP. 
[0034] While immersive picture P_3 is depicted "inside" 
immersive picture P„2 , in effect immersive pictures P_2 and P_3 
are co- located. However, in the present embodiment, only one of 
simultaneously played immersive pictures P__2 and P_3 will be 
displayed to a user at any given time. 

[0035] Figure 2B is a two-dimensional representation of the 
coverage of two 3 60° immersive pictures P_2 and P„3 in accordance 
with the embodiment of Figure 2A. Immersive pictures P_2 and P„3 
are two-dimensional so that they may be stored in conventional 
two-dimensional memory. Immersive picture P_2 is made two- 
dimensional by separation along seam S_2 . Similarly, immersive 
picture P_3 is made two-dimensional by separation along seam S_3 . 
As shown, an overlap distance OVERLAP is the distance between 
edge E5 (at seam S_3 in Figure 2A) and edge E4 (at seam S„2 in 
Figure 2A) , which represents the amount of overlap between the 
seams of immersive pictures P_2 and P_3 . 

[0036] Immersive pictures P_2 and P_3 may be applied to a 
standard display software program to provide interactivity with a 
user. The standard display software program provides a view 
window 201, which effectively defines the user's field of view. 
Thus, the portion of immersive picture P_2 or P_J3 that is visible 
to a user is that portion of the picture bounded by view window 
201. Cursor 202 provides the control mechanism for the user to 
pan around immersive picture P_2 or P_3 . 

[0037] Figures 3A-3C are two-dimensional representations of 
the coverage of two 360° immersive pictures P_2 and P_3 in 
accordance with the embodiment of Figure 2B. As shown, the 
overlap distance OVERLAP -is the distance between edge E5 and edge 
E4, which represents the amount of overlap between seams S„2 and 
S_3 (Figure 2A) . Cursor 202, which is located towards the edge 
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E4 side of view window 2 01, causes view window 2 01 to pan towards 
edge E4 . In response, view window 201 moves in relation to 
immersive picture P_2 as shown in Figure 3B. 

[0038] Figure 3B shows view window 201 located in the area of 
overlap between edges E4 and E5 . To prevent real-time seam 
distortion from appearing in view window 2 01, a distance Di„E4 is 
defined relative to edge E4 such that when view window 201 is 
panning toward edge E4 and reaches the distance Di„E4 from edge 
E4, view window 2 01 will cease displaying immersive picture P_2 
and will instead display immersive picture P_3 (Figure 3C) . 
Because immersive picture P„3 is identical to immersive picture 
P_2 except that seam S_„3 (Figure 2A) of immersive picture P_3 is 
located in a different portion of immersive picture P__3 relative 
to the picture content than seam S„2 of immersive picture P_2 
(Figure 2 A) , the picture shown to the user through view window 
201 will be free of real-time seam distortion. That is, rather 
than showing a portion of immersive picture P„2 including seam 
S_2 (Figure 2A) , a portion of immersive picture P_3 (having 
identical content but no seam) is shown. 

[0039] Similar distances Di_E3, D 2 _E5, and D 2 _E6 are defined 
such that when view window 201 is panning towards edges E3, E5, 
and E6, respectively, the picture shown through view window 2 01 
is changed when reaching that distance from the respective edge 
to prevent display of the seam of a picture. The overlap 
distance OVERLAP is greater than the length of view window 201 
plus Di_E4 plus D 2 _E5 as well as greater than the length of view 
window 201 plus Di__E3 plus D 2 _E6 to allow for proper transition of 
pictures. In this way, real-time seam distortion is eliminated 
from the user's field of view by the simultaneous use of two 
identical pictures having different seam locations. 
[0040] Figure 4A is a representation of an immersive picture 
P_4 that is an environment map greater than 360°. For example, 
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immersive picture P„4 may be 390°, having 30° of overlapping 
picture content, or 420°; having 60° of overlapping picture 
content. The field of view in immersive picture P_4 shows a tree 
TREE 2 , a full house H0USE3 , a full house H0USE4, and a house 
portion HOUSE3„A. As described above, because memory is arranged 
in a two-dimensional array, immersive picture P„4 is stored as a 
two-dimensional array in memory. Because the picture content is 
greater than 3 60°, some objects represented within immersive 
picture P_4 are repeated. For example, the rightmost portion of 
full house H0USE3 is repeated as house portion HOUSE3_A. In 
displaying immersive picture P_4, the two-dimensional 
representation of Figure 4A is converted to a cylindrical 
representation . 

[0041] Figure 4B is a cylindrical representation of immersive 
picture P_4 of Figure 4A. Immersive picture P_4 near edge E8 
depicts full house H0USE3 and tree TREE 2 . House portion H0USE3_A 
is depicted near edge E7 of immersive picture P_4 . Full house 
HOUSE2 is shown around the back side of the cylinder. An overlap 
distance OVERLAP 2 represents the amount of overlap in picture 
content between edges E7 and E8 . Thus, if immersive picture P_4 
is 390°, having 3 0° of overlapping picture content, then the 
overlap distance OVERLAP2 is 3 0°. The content of immersive 
picture P_4 in the area from edge E7 a distance back along 
immersive picture P__4 is repeated in the area from edge E8 a 
distance forward along immersive picture P_4 . While Figure 4B 
depicts immersive picture P__4 as being split along the overlap 
distance OVERLAP2 for clarity, the overlapping picture content is 
instead essentially co-located. 

[0042] Figures 4C-4E are cylindrical representations of 
immersive picture P_4 of Figure 4B at various angles of view. A 
view window 401 displays the portion of the picture content of 
immersive picture P_4 that is bordered by view window 401. Thus, 
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Figure 4C depicts view window 401 at a first point in time; at 
which time view window 401 depicts the content of immersive 
picture P_4 near edge E7 . As a result, view window 401 depicts a 
portion of house portion H0USE3_A. As view window 401 is moved 
towards edge E7 , a point is reached where the content within the 
boundaries of view window 401 is repeated near the edge E8 side 
of immersive picture P_4 . At this point, view window 401 may 
display that content from the portion of immersive picture P_4 
near edge E7 or from the portion of immersive picture P_4 near 
edge E8 . Therefore, to prevent view window 401 from reaching 
edge E7 of immersive picture P_4, the portion of the picture 
content of immersive picture P_4 is changed from the portion near 
edge E7 to the portion near edge E8. Specifically, view window 
401 changes from depicting a portion of house portion H0USE3_A to 
depicting a portion of full house HOUSE3 . This change in view 
window content is shown more clearly in Figure 4D. 
[0043] Figure 4D depicts view window 401 at a second point in 
time, at which time view window 401 depicts the contents of 
immersive picture P_4 near edge E8 . As a result, view window 
depicts a portion of full house HOUSE3 . As view window 401 moves 
away from edge E8 (i.e. towards edge E7) the content of immersive 
picture P„4 bordered by view window 401 changes. Figure 4E 
depicts view window 401 at a third point in time, at which time 
view window 401 depicts another portion of full house HOUSE 3 and 
a portion of tree TREE2 . 

[0044] Figures 4F and 4G are two-dimensional representations 
of the coverage of immersive pictures P__4 in accordance with the 
embodiment of Figure 4A. Figure 4F shows view window 401 located 
in the area of repeated picture content near edge E7 . To 
traversing edge E7 within view window 401, a distance Di_E7 is 
defined relative to edge E7 such that when view window 401 is 
panning toward edge E7 and reaches the distance Di_E7 from edge 
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E7, view window 401 will cease displaying the portion of 
immersive picture P_4 near edge E7 and will instead display the 
repeated portion of immersive picture P_4 near edge E8 as 
described with respect to Figures 4C and 4D. Because the content 
of immersive picture P_4 is repeated near edges E7 and E8, the 
picture shown to the user through view window 401 will not cross 
an edge of immersive picture P„4 (and thus is free of real-time 
seam distortion) . 

[0045] Figure 4H is a two-dimensional representation of a time 
sequenced immersive video in accordance with the embodiment of 
Figure 4A. Movie M0VIE_4 includes M (e.g. M = 30) sequential 
immersive pictures, immersive pictures P_4_1-P_4_M. Immersive 
picture P_4_2 is one time step (e.g. one- thirtieth of a second) 
behind immersive picture P__4„l (i.e. immersive picture P__4, 
Figure 4A) . Similarly immersive picture P_4_3 is one time step 
behind immersive picture P_4_2 . In one embodiment, movie MOVIE_4 
is comprised of self-contained sequential bitmaps. 
[0046] Similar to Figures 4C-4G, view window 401 pans around 
movie MOVIE„4 in response to user input. However, because movie 
M0VIE_4 is comprised of a series of sequential pictures, each 
time step a different, time related picture is shown in view 
window 201. In other words, while the user is panning within 
movie M0VIE_4, the user is actually panning through time as well 
as around a picture. For example, in the first time period a 
first portion of immersive picture P_4_l is shown. Panning 
towards edge E8_l the first time period later, view window 401 
will contain the portion of immersive picture P„4_2 in the 
direction of edge E8_l of immersive picture P_4_l . This example 
is shown more clearly in Figures 41 and 4 J . 

[0047] Figure 41 is the first in a series of sequential 
pictures for movie M0VIE_4 in accordance with the embodiment of 
Figure 4H. Cursor 4 02 is causing view window 401 to pan down and 
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towards edge E8_l of immersive picture P_4_l of movie M0VIE„4 . A 
first time period later, view window 401 has moved in the 
direction of edge E8„l. However, because a movie rather than a 
single picture is displayed, the actual picture displayed through 
view window 401 is immersive picture P„4_2 of movie MOVIE__4 . 
Thus, panning has occurred both within a picture (moving through 
immersive picture P_4_l while it is displayed) and through time 
(continuing to pan through immersive picture P_4„2 when it is 
displayed in place of immersive picture P_4_l) . 

[0048] To prevent real-time seam distortion from appearing in 
view window 401, a distance Di_E7 is defined relative to edges 
E7_J_-E7_3, similarly to that described for Figures 4F and 4G, 
such that when view window 401 is panning toward edge E7_2 and 
reaches the distance Di_E7 from edge E7_2 , view window 701 will 
move to display the repeated content near edge E8„2 . Because the 
content is repeated near the edges in immersive picture P„4„2 , 
the picture shown to the user through view window 401 will be 
free of real-time seam distortion. In this way, real-time seam 
distortion is eliminated from the user's field of view by the 
simultaneous use of two identical movies having different seam 
locations . 

[0049] Figure 5 is a two-dimensional representation of a two 
time sequenced 360° immersive videos in accordance with the 
embodiment of Figure 2A. Movies M0VIE„1 and M0VIE„2 include N 
(e.g. N = 30) sequential immersive pictures each, immersive 
pictures P_2_1-P_2_N and P_3_l-P_3 _N, respectively. Immersive 
picture P„2„2 is one time step (e.g. one- thirtieth of a second) 
behind immersive picture P_2_l (i.e. immersive picture P_2 , 
Figure 2A) . Similarly immersive picture P_2_3 is one time step 
behind immersive picture P__2__2 . Immersive picture P__3_2 is one 
time step (e.g. one- thirtieth of a second) behind immersive 
picture P_3_l (i.e. immersive picture P_3 , Figure 2A) . Immersive 
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pictures P_2_3-P_2_N and P_3_2-P_3_N are similarly related in 
time. In one embodiment, movies M0VIE_1 and M0VIE_2 are 
comprised of self-contained sequential bitmaps. 

[0050] Similar to Figures 3A-3C, view window 201 pans around 
movies M0VIE_1 and M0VIE_2 in response to user control of cursor 
202. However, because movies M0VIE_1 and M0VIE_2 are comprised 
of a series of sequential pictures, each time period a different 
time-related picture is shown in view window 201. In other 
words, while the user is panning within movie M0VIE_1, the user 
is actually panning through time as well as around a picture. 
For example, in the first time period a first portion of 
immersive picture P_2_l is shown. Panning towards edge E4_l the 
first time period later, view window 201 will contain the portion 
of immersive picture P_2_2 in the direction of edge E4 of 
immersive picture P_2_l . This example is shown more clearly in 
Figures 5A-5C. 

[0051] Figure 6A is the first in a series of sequential 
pictures for movies MOVIE_l and M0VIE_2 in accordance with the 
embodiment of Figure 5. Cursor 202 is causing view window 201 to 
pan towards edge E4_l of immersive picture P_2_l of movie 
MOVIE_l. A first time period later, view window 201 has moved in 
the direction of edge E4_l . However, because a movie rather than 
a single picture is displayed, the actual picture displayed 
through view window 201 is immersive picture P_2_2 of movie 
M0VIE_1. Thus, panning has occurred both within a picture 
(moving through immersive picture P_2_l while it is displayed) 
and through time (continuing to pan through immersive picture 
P_2_2 when it is displayed in place of immersive picture P_2_l) . 
[0052] To prevent real-time seam distortion from appearing in 
view window 201, a distance Di_E4 is defined relative to edges 
E4_l-E4_3 such that when view window 2 01 is panning toward edge 
E4_2 and reaches the distance Di_E4 from edge E4_2, view window 
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201 will cease displaying immersive picture P_2_2 and will 
instead display immersive picture P_3_2 (Figure 6C) . Because 
immersive picture P_3_2 is identical to immersive picture P_2_2 
except that the seam of immersive picture P_3_2 is located in a 
different portion of immersive picture P_3_2 than the edge of 
immersive picture P_2_l (similar to Figure 2A) , the picture shown 
to the user through view window 201 will be free of real-time 
seam distortion. Similar distances are defined relative to other 
edges for the other pictures in movies M0VIE_1 and MOVIE_2 
(Figure 5) . In this way, real-time seam distortion is eliminated 
from the user's field of view by the simultaneous use of two 
identical movies having different seam locations. 
[0053] In one embodiment, one of both sets of pictures 
comprising movies M0VIE_1 and MOVIE_2 contain less than a 360 
degree field of view. In this embodiment, the seams of movies 
M0VIE_2 are offset from the seams of movie MOVIE_l by at least 
the width of the view window. 

[0054] Appendix I, found at the end of the present document, 
is a sample code for implementing an embodiment of the present 
invention in the Macromedia™ Flash standard display software. 
[0055] Figure 7 is a two-dimensional representation of a two 
time sequenced immersive videos in accordance with an embodiment 
of the present invention. Movie Movie_5 is a 360° immersive video 
and movie M0VIE_6 is a M6_WIDTH immersive video, where M6_WIDTH 
is twice the width of view window 701. Because movie M0VIE_6 is 
twice the width of view window 7 01, movie M0VIE_6 can be 
displayed in place of movie M0VIE_5 in the vicinity of the seam 
formed by edges E5_l and E6_l, thereby eliminating the need to 
generate seams in movie M0VIE_5 real-time. Movies M0VIE_5 and 
M0VIE_6 include N (e.g. N = 30) sequential immersive pictures 
each, immersive pictures P_5_1-P_5_N and P_6_1-P_6_N, 
respectively. Immersive picture P_5_2 is one time step (e.g. 
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one-thirtieth of a second) behind immersive picture P_5_l (i.e. 
immersive picture P_2 , Figure 2A) . Because each picture P_6_JL- 
P_6_N in movie M0VIE_6 is smaller than each picture P_5__1-P_5_N 
in movie M0VIE_5, movie M0VIE_6 beneficially requires less memory 
for storage and playback. 

[0056] Figure 8 is a block diagram of a system 800 
implementing an immersive video display system in accordance with 
an embodiment of the present invention. System 800 includes a 
first movie memory 801 and a second movie memory 802 for storing 
movies. In one embodiment, the movies are a video stream. Movie 
Selector 803 selects a movie to be displayed, choosing between 
simultaneously playing movies. 

[0057] Once a movie has been selected for display, View Window 
Contents Selector 804 determines which portion of the displayed 
movie will appear in the field of view of the user. That portion 
is displayed in View Window Display 805. User Interface 807 
provides control of the field of view to the user. Thus, User 
Interface 807 (e.g. mouse or joystick) allows the user to pan the 
view window around the displayed movie. Seam Detector 806 
determines when the view window reaches a transition edge (e.g. a 
distance D!__E4 from edge E4_2 in Figure 6B at which the view 
window changes from displaying a portion of one movie to 
displaying a portion of another movie) of the currently displayed 
movie. When the user pans to a transition edge of the currently 
displayed movie, Controller 808 is alerted to change the selected 
movie. Thus, Controller 808 signals Movie Selector 803 to 
display a different simultaneously running movie. In this way, 
the user is allowed panning access to movies without seam 
distortion appearing in the field of view of view window display 
805. 

[0058] The various embodiments of the structures and methods 
of this invention that are described above are illustrative only 
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of the principles of this invention and are not intended to limit 
the scope of the invention to the particular embodiments 
described. For example, in view of this disclosure, those 
skilled in the art can define other curved surfaces that are 
stored in two-dimensional memory, such as a sphere and so forth, 
and use these alternative surfaces to create a method or system 
according to the principles of this invention. Thus, the 
invention is limited only by the following claims. 
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Macromedia™ Flash Implementation of One Embodiment 
Main 

actions for frame 1 

tmi="2" ; 

stop ( ) ; 
actions for frame 2 

startDrag( "dragControl", true) ; 

stop ( ) ; 
dragControl , ( dragControl ) 
Play Demo Button 

actions for Play Demo Button 

on (release) { 
play ( ) ; 

} 

logo 
low-res 

ENROUTE INC., (Times, 24 pts) 
slide object, (nav) 
Symbol Definition (s) 
dragControl 

actions for frame 1 
speeds 4 5 ; 

drag=" /dragcontrol " ; 
/ : w=get Property ( tt /nav" , _width) ; 
/ :xPos=getProperty (drag,_x) ; 
/ :yPos=getProperty (drag,„y) ; 
/ :xPos=/ rxPos- (400/2) ; 

setProper ty ( " /nav" , _x , getProperty ( " /nav" , _x) 

- ( / :xPos/ speed) ) ; 
if (Number (getProperty ( " /nav" , _x) ) < 

Number (-(/ :w/2) ) ) { 
setProperty( u /nav" ,__x, 0.01) ; 
} e i S e if (Number (getProperty ("/nav", _x) )>0 { 

setProperty ("/nav" ,_x, -/ :w/2) ; 

} 

actions for frame 2 
gotoAndPlay (__currentf rame-1) ; 
Play Demo Button 
logo 
low-res 
slide object 

actions for frame 1 
m=0; 

while (Number (m) <Number ( (/ :tmi*2) -1) ) { 
m=Number (m) +1 ; 
dup="butn" add m; 

duplicateMovieClip ( "butn" , dup, Number (m) +10 ) ; 
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set Property (dup , _x, Number (get Property (dup , _x) ) 
+Number (m*getProperty (dup, _width) ) 

} 

Immersive Video, (butn) 
Immersive Video 

Sequenced Images 
Sequenced Images 

APPENDIX I 
Copyright (c) 2 001 Enroute Inc. 
All Rights Reserved 
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